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This issue o£ FEEDBACK, a newsletter produced by the 
the Austin Independent School District 0££ice o£ Research and 
Evaluation (ORE), illustrates the accuracy, validity, and £airnes8 o£ 
ORE reports. The independence o£ the reports is explained. Internal 
and external quality controls are used to ensure reliability and 
accuracy o£ the reports. National dissemination o£ the information 
has been accomplished through publication and presentations at the 
annual meetings of the American Educational Research Association 
(AERA) . The ORE has established a national reputation and its staff 
has been represented in the elective offices and appointments of the 
AERA school evaluation division. Technical documentation of research 
is available in technical reports produced to support the findings in 
the general reports. (DWH) 
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Can I Trust ORE Reports? 

o 

O I'or eleven years now, the Office of Research and Evaluation (ORE) has 
^ published reports for use in decision making. We in ORE know the 

quality and validity of those reports and present them with a high level 
C\J of confidence to our audiences. However, our audiences often do not 
, , , know the quality control behind a report and may have questions regard- 
ing the level of trust they should place in the findings and conclusions 
presented. Are ORE reports accurate, valid, and unbiased? 

As a user of ORE reports, please consider the following information 
when judging our work. 
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INDEPENDENCE 



ORE reports directly to the Superintendent, not the in- 
structional or administrative staff whose programs are 
being evaluated. Our reports are published and pro- 
vided to the Board of Trustees without editing by the 
instn ctional and administrative staff. However, our 
report drafts are reviewed by them, and comments on 
accuracy and comprehensiveness are carefully considered. 
If disagreements exist with the findings within an ORE 
report, non-ORE staff may prepare a separate statement 
accompany the report when it is dioseminated. 



All published ORIi reports are provided to the Board 
of Trustees, placed in the AISD Professional Library, 
and distributed to other audiences as appropriate or 
requested . 
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RELIABILITY AND ACCURACY 

Internal quality control is a primary concern for 
ORE, Our data, analyses, and reports are checked 
carefully at each step. ORE staff review each 
others' analyses and reports as a standard proce- 
dure. An important aspect of internal quality con- 
trol is the use of data across evaluations; conse- 
quently, one person's data are carefully validated 
as others use them for additional analyses* Be- 
cause of the longitudinal nature of ORE's data 
bases and reports, data are reanalyzed and re- 
checked annually as a new year's crop of informa- 
tion is added. 

A critical external accuracy check occurs as re- 
searchers from other agencies use ORE data. Re- 
searchers from The University of Texas, the Na- 
tional Institute of Education, the National Center 
for the Study of Vocational Education at Ohio State 
Uni versity, and other agencies have reanalyzed our 
data. Recently, a researcher from Rochester, New York, independently recalculated and 
verified some of our districtwide achievement score averages as part of a study of re- 
porting procedures. 

ORK reports and supportive d.-t-a are available to other researchers for use or verifi- 
cation, and we welcome those .^jportunitres . 





NATIONAL DISSEMINATION 

ORE staff have published our findings and 
procedures in professional books and jour- 
nals and have been a top contributor among 
public school research and evaluation of- 
fices to the programs at the annual meeting 
of the American Educational Research Asso- 
ciation (AERA) over the past five years. 
This represents an excellent record of ac- 
ceptance by our peers in research and eval- 
uation who judge and accept these publica- 
tions and presentations. 



NATIONAL RECOGNITION 

ORi; staff have been well represented in the 
elective offices and appointments of the 
school evaluation division of AHRA. Six na- 
tional awards have been given to ORI-; publica- 
tions by this division. Currently, Glynn 
Ligon is the ARRA acquisition editor for 
competency testing issues, and Walter Jordan- 
Davis is the graduate student representative 
for the evaluation division. 
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A key measure of ORE's national reputation is the high number of requests received for 
copies of our reports. In addition, other evaluators and researchers from all parts 
of the country call, write, and visit for advice and expertise in a variety of areas. 
Our office is known to have one of the best longitudinal data bases of any public 
school system, and researchers often contact us for data related to their work. 



PROFESSIONAL STAFF 

ORE staff became qualified to conduct our sophisticated research and evaluation ac- 
tivities through graduate programs in research, psychology, education, and related 
areas. Ph.D. -level training is required or preferred for all evaluator positions. 
Six of our current staff members are former public school teachers, and five others 
have taught at the college level. 

With backgrounds in research and a commitment to contributions to the literature, ORE 
staff members subscribe to the highest of professional standards including the Stan- 
dards for Evaluations of Educational Programs, Projects, and Materials prepared by 
The Joint Committee on Standards for Educational Evaluation. (This Joint Committee 
is made up of one member from each of 12 national organizations associated with educa- 
tional research.) 



Subscriptions to professional publications and participation in professional organi- 
zations keep ORE staff up to date with the state of the art in research and evalua- 
tion. 
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ADVISORY GROUPS 

ORE has an Evaluation Advisory Committee composed of teachers, principals, administra- 
tors, paronts, citizens, and other researchers. This committee reviews activities and 
advises ORE on our work. In addition meetings are held as needed with elementary, 
junior high, and senior high school building test coordinator groups to review testing 
policies, procedures, and materials. 

ORE often meets with other elementary and secondary advisory groups, faculties, coun- 
selors, PTA's, etc. to elicit ideas and suggestions. 



TECHNICAL DOCUMENTATION 



Replicability is a cornerstone 
of research. All procedures, 
analyses, and data are docu- 
mented in a technical report 
to support the findings in our 
more general report?. Those 
statistics researchers love to 
see (i.e. , F's, t's, r's p's, 
alphas, etc.) are provided in 
our technical reports. 




FEEDBACK is a newsletter of the Office of Research and Evaluation of the 
Austin Independent School District, Austin, Texas. The publication num- 
ber is 83.14. For more information on ORE reports call CRE at 458-1227. 




